Remote robot system and method of controlling remote robot system

ABSTRACT

A remote robot system and a method of controlling a remote robot system capable of responding appropriately according to a user are provided. A remote robot system includes: a robot configured to perform a predetermined operation including collection of local information near the robot; a guide terminal capable of remotely operating the operation of the robot; and a participant terminal capable of remotely communicating with the guide terminal, in which the participant terminal includes: a display panel configured to output the local information acquired from the robot to the user; and an interest detection unit configured to detect user&#39;s interest in the output local information.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese patent application No. 2021-068229, filed on Apr. 14, 2021, thedisclosure of which is incorporated herein in its entirety by reference.

BACKGROUND

The present disclosure relates to a remote robot system and a method ofcontrolling the remote robot system.

Recent techniques have enabled autonomous travelling robots to beremotely operated by users using operation terminals. Further, atelepresence robot including, for example, a camera, a microphone, and amonitor, which is one example of the above robots, enables users tocommunicate with each other remotely or enjoy virtual experiencesremotely. Further, in view of a recent spread of remote services such asWeb meeting services, it is expected that robots that can be remotelyoperated or can be used for remote communication will be used in a broadrange.

With regard to operations performed by users, for example, JapaneseUnexamined Patent Application Publication No. 2016-224583 discloses atechnique in which an analysis server apparatus analyzes the behavior ofa user viewing Web pages by a heat map. Further, Japanese UnexaminedPatent Application Publication No. 2014-50465 discloses a technique inwhich an image processing apparatus that displays a medical image suchas an X-ray CT image designates an area that a user is interested in tomake a diagnosis.

SUMMARY

By using a robot such as a telepresence robot, a remote sightseeing tourwhich enables users to remotely join and virtually experience this tourcan, for example, be provided. In a remote robot system for a remotesightseeing tour, for example, a tour guide (an operator) remotelyoperates a robot and users (participants) who join this tour remotelycommunicate with the guide. In this remote robot system, however, it isdifficult to respond appropriately according to the users since theguide and the users are present in places away from each other. Thetechnique disclosed in Japanese Unexamined Patent ApplicationPublication No. 2016-224583 is to analyze Web page views and thetechnique disclosed in Japanese Unexamined Patent ApplicationPublication No. 2014-50465 is to specify a diagnosis area of a medicalimage. Neither of them solves the above-described problem.

The present disclosure has been made in order to solve theaforementioned problem and provides a remote robot system and a methodof controlling the remote robot system capable of respondingappropriately according to a user.

A remote robot system according to one aspect of the present disclosureincludes: a robot configured to perform a predetermined operationincluding collection of local information near the robot; a first remoteterminal capable of remotely operating the operation of the robot; and asecond remote terminal capable of remotely communicating with the firstremote terminal, in which the second remote terminal includes: aninformation output unit configured to output the local informationacquired from the robot to a user; and an interest detection unitconfigured to detect user's interest in the output local information.Accordingly, the user's interest detected by the second remote terminalcan be detected, whereby it becomes possible to recognize the user'sinterest and respond appropriately according to the user even when thefirst remote terminal is away from the second remote terminal.

Further, the interest detection unit may detect an operation state bythe user in response to the output local information and detect theuser's interest based on the operation state that has been detected.Further, the operation state may include an operation position, thenumber of times of operations, or an operation time in the user'soperation performed on a display screen that displays the localinformation. Accordingly, it is possible to accurately detect theinterest in accordance with the state in which the user has operated.

The interest detection unit may further detect an utterance state of theuser in response to the output local information and detect the user'sinterest based on the utterance state that has been detected. Further,the utterance state may include a sound pressure of a voice emitted bythe user, an utterance time of the voice, or results of recognizing thevoice. Accordingly, it is possible to accurately detect the interest inaccordance with the state of the voice emitted by the user.

The interest detection unit may further detect an attention state by theuser in response to the output local information and detect the user'sinterest based on the attention state that has been detected. Further,the attention state may include the orientation of the face of the user,the orientation of the line of sight of the user, an attention positionof the user, or an attention time of the user. Accordingly, it ispossible to accurately detect the interest in accordance with the statein which the user is focusing on.

The second remote terminal may further include a notification unitconfigured to notify the first remote terminal of interest informationindicating the detected interest. Accordingly, it is possible todefinitely send feedback on the user's interest detected by the secondremote terminal to the first remote terminal.

The first remote terminal may include an interest output unit configuredto output the interest information that has been sent. Further, theinterest output unit may highlight a position that corresponds to theinterest information on a display screen that displays the localinformation. The interest output unit may further display a temporalchange of the interest information on a display screen that displays thelocal information. This enables the first remote terminal to instantlyrecognize the user's interest detected by the second remote terminal.

Further, the first remote terminal may include a remote controldetermination unit configured to determine a method of remotelyoperating the robot based on the interest information that has beensent. Further, the remote control determination unit may determine amethod of moving a robot including a moving direction, a moving path, ora moving speed of the robot based on the interest information. Theremote control determination unit may further determine an informationacquisition method of the robot including an imaging direction of animage of the robot or a voice collection direction based on the interestinformation. Accordingly, it is possible to remotely control a robotappropriately in accordance with the user's interest detected by thesecond remote terminal.

A method of controlling a remote robot system according to one aspect ofthe present disclosure is a method of controlling a remote robot systemincluding: a robot configured to perform a predetermined operationincluding collection of local information near the robot; a first remoteterminal capable of remotely operating the operation of the robot; and asecond remote terminal capable of remotely communicating with the firstremote terminal, in which the second remote terminal outputs the localinformation acquired from the robot to a user, and detects user'sinterest in the output local information. Accordingly, the user'sinterest detected by the second remote terminal can be detected, wherebyit is possible to recognize the user's interest and respondappropriately according to the user even when the first remote terminalis away from the second remote terminal.

According to the present disclosure, it is possible to provide a remoterobot system and a method of controlling the remote robot system capableof responding appropriately according to a user.

The above and other objects, features and advantages of the presentdisclosure will become more fully understood from the detaileddescription given hereinbelow and the accompanying drawings which aregiven by way of illustration only, and thus are not to be considered aslimiting the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a configuration diagram showing a configuration example of aremote robot system according to a first embodiment;

FIG. 2 is an external perspective view showing an external configurationexample of a robot according to the first embodiment;

FIG. 3 is a block diagram showing a functional configuration example ofthe robot according to the first embodiment;

FIG. 4 is a block diagram showing a functional configuration example ofa guide terminal according to the first embodiment;

FIG. 5 is an image view showing a configuration example of a displayscreen of the guide terminal according to the first embodiment;

FIG. 6 is a block diagram showing a functional configuration example ofa participant terminal according to the first embodiment;

FIG. 7 is an image view showing a configuration example of a displayscreen of the participant terminal according to the first embodiment;

FIG. 8 is a sequence diagram showing an operation example of a remoterobot system according to the first embodiment;

FIG. 9 is an image view showing an example of screen display of theguide terminal according to the first embodiment;

FIG. 10 is an image view showing an example of screen display of theguide terminal according to the first embodiment;

FIG. 11A is an image view showing an example of screen display of theguide terminal according to the first embodiment;

FIG. 11B is an image view showing an example of screen display of theguide terminal according to the first embodiment;

FIG. 12 is an image view showing an example of screen display of theguide terminal according to the first embodiment;

FIG. 13 is a block diagram showing a functional configuration example ofa guide terminal according to a second embodiment; and

FIG. 14 is a sequence diagram showing an operation example of a remoterobot system according to the second embodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, with reference to the drawings, embodiments of the presentdisclosure will be described. Throughout the drawings, the same elementsare denoted by the same reference symbols and overlapping descriptionsare omitted as appropriate. While the present disclosure will bedescribed with reference to the embodiments, the present disclosure setforth in the claims is not limited to the following embodiments.Further, not all the components described in the following embodimentsare necessary for solving problems.

First Embodiment

First, a first embodiment according to the present disclosure will bedescribed. FIG. 1 shows a configuration example of a remote robot system10 according to this embodiment. As shown in FIG. 1 , the remote robotsystem 10 according to this embodiment includes a robot 100, a guideterminal (first remote terminal) 200, a plurality of participantterminals (second remote terminals) 300 (e.g., 300-1 to 300-3), and asystem server 400.

The remote robot system 10 is a system in which a guide (tour guide) Gremotely operates the robot 100 located in a tourist spot (a firstenvironment) by operating the guide terminal 200 and participants P areable to virtually experience a sightseeing tour via the robot 100 thathas been remotely operated. In the remote robot system 10, a pluralityof participants P (e.g., P1 to P3) and the guide G are connected to therobot 100, which is, for example, a telepresence robot, via terminalapparatuses, and the plurality of participants P are able to enjoy theremote sightseeing tour via the robot 100 while sharing videoimages/voices acquired by the robot 100. One of the users connected tothe robot 100 is the guide G, who gives a tour of the tourist spot whilechanging the travelling direction or the angle of view of the robot byremotely operating the robot 100. The participants P enjoy sightseeingfrom remote places while watching and listening to the videoimages/voices of the robot 100 and the guide G. In this tour, the guideG needs to give a tour of the tourist spot in accordance with interestof the participants P, but it is difficult for the guide G toappropriately give the tour on his/her connection terminal in accordancewith the interest of the participants P because the guide G cannot seethe reactions of the participants P due to the distance between theirrespective positions. In order to solve this problem, in thisembodiment, interest of the participants P is detected, and the detectedinterest is further fed back to the guide G, which enables the guide Gto guide the tour in accordance with the interest of the participants P.

Note that a plurality of robots 100 may be installed so that the remoteoperation and the virtual experience can be provided for the respectiveparticipants. Further, a plurality of guide terminals 200 may beincluded so that a plurality of guides G may be able to remotely operatethe plurality of robots 100.

As shown in FIG. 1 , the robot 100, the guide terminal 200, and theplurality of participant terminals 300 are located in environments awayfrom one another (a first environment, a second environment, and a thirdenvironment) and are connected to the system server 400 via wirelessrouters 600 and the Internet 500. For example, the system server 400provides data, processing, services and the like that are necessary forthe remote operation or remote communication for the robot 100, theguide terminal 200, and the participant terminals 300. For example,while the wireless routers 600, the robot 100, the guide terminal 200,and the participant terminals 300 can communicate with one another via awireless LAN, they may communicate with one another by mobile phonecommunication such as Long Term Evolution (LTE) or 5th Generation MobileCommunication System (5G). By using mobile phone communication for thecommunication of the robot 100, the robot 100 can be used in a largenumber of places, whereby it becomes possible to remotely operate robots100 installed in various places. In this case, the wireless routers 600may be base stations of mobile phone communication.

The robot 100 is able to execute a predetermined operation includingcollection of local information (including, for example, images orvoices) near the robot 100 in the first environment. The robot 100 isremotely operated by the guide G, who is a remote operator and is in thesecond environment located away from the first environment, operatingthe guide terminal 200 (operation terminal) via the system server 400connected to the Internet 500. The robot 100 receives various kinds ofoperation signals from the guide terminal 200 and executes, for example,a moving operation and/or an imaging operation. Further, the guide G andthe participants P are able to remotely communicate with each other viathe robot 100.

The robot 100 captures images of a space near the robot 100 by a camera131 (an imaging unit) and transmits the captured images (video images)to the guide terminal 200 and the participant terminals 300 via theInternet 500. Further, a microphone 132 collects voices emitted near therobot 100, and transmits the collected voices to the guide terminal 200and the participant terminals 300 via the Internet 500. Further, therobot 100 receives voices from the guide terminal 200 and theparticipant terminals 300 via the Internet 500 and outputs the receivedvoices from the speaker 133. The robot 100 is able to execute a graspingmotion or the like performed by a hand 124 in accordance with anoperation performed by the guide terminal 200. It is therefore possibleto provide various kinds of virtual experience in a sightseeing tour.

While the Internet 500 is used as a network to which the robot 100, theguide terminal 200, the participant terminals 300, etc. are connected inthis embodiment, the network may be other types of network such as anintranet. Alternatively, the robot 100, the guide terminal 200, and theparticipant terminals 300 may be directly connected to each otherthrough near-field communication such as Bluetooth (RegisteredTrademark) without using any network interposed therebetween. That is,the remote control and the remote communication may be performed withoutusing the Internet 500 and the system server 400 interposedtherebetween.

The guide terminal 200, which is a terminal operated by the guide G inthe second environment, remotely operates the robot 100 and performsremote communication with the participant terminals 300 and the robot100. The participant terminals 300 (e.g., 300-1 to 300-3), which areterminals operated by participants P (e.g., participants P1 to P3) inthe third environment (e.g., environment 3-1 to environment 3-3),perform the remote operation with the guide terminal 200, otherparticipant terminals 300, and the robot 100.

The guide terminal 200 and the participant terminals 300 are, forexample, tablet terminals and include display panels 201 and 301 onwhich touch panels are superimposed thereon. The display panels 201 and301 are able to display the captured images received from the robot 100and the guide G and the participants P can visually recognize asituation in the spatial area near the robot 100 in an indirect manner.In other words, the display panel 201 is an information output unitconfigured to output local information of the robot 100 to the guide Gand the display panel 301 is an information output unit configured tooutput local information of the robot 100 to the participants P (user).

Further, the guide terminal 200 and the participant terminals 300 areable to output voices received from the robot 100, and the guide G andthe participants P can listen to the voices emitted near the robot 100in an indirect manner. The guide G is able to cause various kinds ofoperation signals for operating the robot 100 to be generated byoperating the guide terminal 200 via the touch panel. The generatedoperation signal is transmitted to the robot 100 via the Internet 500.

Further, the participant terminal 300 includes an interest detectionunit 321 that detects interest of the participants P in the localinformation of the robot 100 that has been output. The interestdetection unit 321 generates interest information indicating theinterest of a participant P in accordance with the operation of thedisplay screen of the display panel 301 by the participant P via thetouch panel. The generated interest information is transmitted to theguide terminal 200 via the Internet 500. Further, the guide terminal 200includes the interest analysis unit 221 that analyzes interestinformation received from the participant terminal 300. The interest ofthe participant P that has been analyzed is displayed on the displaypanel 201. The display panel 201 also serves as an interest output unitthat outputs interest of the participant P.

FIG. 2 is an external perspective view showing an example of an externalconfiguration of the robot 100 according to this embodiment. As shown inFIG. 2 , the robot 100 according to this embodiment mainly includes amovable base part 110 and a main-body part 120.

The movable base part 110 supports two driving wheels 111 and a caster112, each of which is in contact with a traveling surface, inside itscylindrical housing. The two driving wheels 111 are arranged so that thecenters of their rotation axes coincide with each other. Each of thedriving wheels 111 is rotationally driven by a motor (not shown)independently of each other. The caster 112 is a driven wheel and isdisposed so that its pivotal axis extending from the movable base part110 in the vertical direction axially supports the wheel at a place awayfrom its rotation axis. Further, the caster 112 follows the movement ofthe movable base part 110 so as to move in the moving direction of themovable base part 110.

Further, the movable base part 110 includes a laser scanner 134 in aperipheral part of its top surface. The laser scanner 134 scans acertain range on the horizontal plane at intervals of a certain steppingangle and outputs information as to whether or not there is an obstaclein each direction. Further, when there is an obstacle, the laser scanner134 outputs a distance to the obstacle.

The main-body part 120 includes, mainly, a body part 121 mounted on thetop surface of the movable base part 110, a head part 122 placed on thetop surface of the body part 121, an arm 123 supported on the sidesurface of the body part 121, and the hand 124 disposed at the tip ofthe arm 123. The arm 123 and the hand 124 are driven by motors (notshown) and grasp an object to be grasped. The body part 121 is able torotate around a vertical axis with respect to the movable base part 110by a driving force of a motor (not shown).

The head part 122 mainly includes the camera 131, the microphone 132, aspeaker 133, and a display panel 141. The camera 131 is disposed, forexample, on the front surface of the head part 122 in such a way thatthe front direction of the head part 122 is an imaging direction. Thecamera 131 may either be a two-dimensional camera or a stereo camera.When, for example, the camera 131 is a stereo camera, this camera 131has a configuration in which two camera units having the same angle ofview are arranged away from each other, and it outputs imaging signalstaken by the respective camera units.

The microphone 132 is disposed, for example, on the front surface of thehead part 122 in such a way that the front direction of the head part122 is a collect direction. The microphone 132 may either be aunidirectional microphone or a non-directional microphone. The speaker133 is disposed, for example, in front or side of the head part 122 insuch a way that the front direction of the head part 122 is an outputdirection. The speaker 133 may either be a monaural speaker or a stereospeaker.

The display panel 141 is disposed on the front surface of the head part122 in such a way that the front direction of the head part is a displaydirection. The display panel 141 is, for example, a liquid crystalpanel, and displays an animated face of a pre-defined character anddisplays information about the robot 100 in the form of text or by usingicons. By displaying the face of the character on the display panel 141,it is possible to impart an impression that the display panel 141 is apseudo face part to people around the robot 100. The display panel 141is able to display images of the guide G and the participants P receivedfrom the guide terminal 200 and the participant terminals 300.

The head part 122 is able to rotate around a vertical axis with respectto the body part 121 by a driving force of a motor (not shown). Thus,the camera 131 can shoot an image in any direction, the microphone 132can collect a sound in any direction, the speaker 133 can output a soundin any direction, and the display panel 141 can show the displayedcontent in any direction.

FIG. 3 is a block diagram showing a functional configuration example ofthe robot 100 according to this embodiment. As shown in FIG. 3 , therobot 100 includes, as functional blocks, besides the display panel 141,the camera 131, the microphone 132, the speaker 133, and the laserscanner 134 described above, a control unit 150, a movable-base driveunit 142, an upper-body drive unit 143, a memory 160, a communicationunit 170, and a GPS unit 135.

Main elements related to the remote operation and the remotecommunication by the guide terminal 200 and the participant terminals300 will be described. However, the robot 100 may include elements inits configuration other than the above ones and may include additionalelements that contribute to the remote operation and the remotecommunication.

The control unit 150, which is, for example, a Central Processing Unit(CPU), executes control of the whole robot 100 and various calculationprocesses by executing a control program read from the memory 160.Further, the control unit 150 also serves as a function execution unitthat executes various calculations and controls related to the control.

The control unit 150 is included in, for example, a control unitdisposed in the body part 121. The movable-base drive unit 142 includesthe driving wheels 111, and a driving circuit and motors for driving thedriving wheels 111. The control unit 150 executes rotation control ofthe driving wheels by sending a driving signal to the movable-base driveunit 142. Further, the control unit 150 receives a feedback signal suchas an encoder signal from the movable-base drive unit 142 and recognizesa moving direction and a moving speed of the movable base part 110.

The upper-body drive unit 143 includes the arm 123 and the hand 124, thebody part 121, the head part 122, and driving circuits and motors fordriving these components. The control unit 150 performs a graspingmotion and a gesture by sending a driving signal to the upper-body driveunit 143. Further, the control unit 150 receives a feedback signal suchas an encoder signal from the upper-body drive unit 143 and recognizespositions and moving speeds of the arm 123 and the hand 124, andorientations and rotation speeds of the body part 121 and the head part122.

The display panel 141 receives an image signal generated by the controlunit 150 and displays an image thereof. The control unit 150 generatesan image signal of the character or the like based on information in thememory 160, received information or the like and causes the displaypanel 141 to display an image thereof. When, for example, the displaypanel 141 includes a touch panel, the display panel 141 outputs a touchsignal to the control unit 150.

The camera 131 shoots, for example, the front part of the robot 100 inaccordance with a request from the control unit 150 and outputs animaging signal generated by imaging to the output unit 150. The controlunit 150 executes image processing using the imaging signal from thecamera 131 and converts the imaging signal into a captured image inaccordance with a predetermined format.

The microphone 132 collects voices emitted near the robot 100 inaccordance with a request from the control unit 150, and outputs a voicesignal generated by sound collection to the control unit 150. Thecontrol unit 150 performs voice processing by using the voice signalfrom the microphone 132 and converts the imaging signal in accordancewith a predetermined format.

The speaker 133 outputs, for example, voices sent from the guideterminal 200 and the participant terminals 300. The control unit 150outputs voice signals received from the guide terminal 200 and theparticipant terminals 300 to the speaker 133 via the communication unit170, and the speaker 133 outputs the voice signals from the control unit150 to the surroundings.

The laser scanner 134 detects whether there is an obstacle in the movingdirection of the robot 100 in accordance with a request from the controlunit 150, and outputs a detection signal, which indicates the results ofthe detection, to the control unit 150. The GPS unit 135 detects theposition of the robot 100 based on a GPS signal received from a GlobalPositioning System (GPS) satellite in accordance with a request from thecontrol unit 150, and outputs a positional signal (positionalinformation), which is the result of the detection, to the control unit150. The position of the robot 100 may be detected by a method otherthan the GPS.

The memory 160, which is a non-volatile storage medium, is, for example,a solid-state drive. The memory 160 stores, besides a control programfor controlling the robot 100, various parameter values, functions,lookup tables and the like used for the control and the calculation. Thememory 160 stores, for example, a map DB 161, which is a database of mapinformation that describes the space of the first environment where therobot 100 is planned to autonomously move. In the map DB 161, target mapinformation which has been acquired from, for example, the system server400 according to the range in which the robot 100 moves is accumulated.

The communication unit 170, which is, for example, a wireless LAN unit,performs radio communication with the wireless router 600. Thecommunication unit 170 transmits or receives control signals, data orthe like to or from the guide terminal 200 and the participant terminals300 via the wireless router 600. For example, the communication unit 170receives a designation signal and an operation instruction sent from theguide terminal 200 and outputs the designation signal or the operationinstruction to the control unit 150. The designation signal is a signalfor requesting the captured image or the voice signal generated in theguide terminal 200 by designating a spatial area that the user wants toobserve. Further, the communication unit 170 transmits captured imagesacquired by the camera 131 and voices acquired by the microphone 132 tothe guide terminal 200 and the participant terminals 300 in accordancewith control by the control unit 150. The communication unit 170 maytransmit positional information detected by the GPS unit 135 to theguide terminal 200 and the participant terminals 300 and transmit mapinformation accumulated in the map DB 161 to the guide terminal 200 andthe participant terminals 300. The communication unit 170 furtherreceives images or voices sent from the guide terminal 200 and theparticipant terminals 300 and outputs these images or voices to thecontrol unit 150.

FIG. 4 is a block diagram showing a functional configuration example ofthe guide terminal 200 according to this embodiment. As shown in FIG. 4, the guide terminal 200 includes, as functional blocks, a calculationunit 220, the display panel 201, an input unit 202, a microphone 203, aspeaker 204, a camera 205, a memory 230, and a communication unit 240.

Main elements related to the remote operation of the robot 100 or theremote communication with the robot 100 and the participant terminals300 will be described. However, the guide terminal 200 may includeelements in its configuration other than the above one and may includeadditional elements that contribute to the processing for performingremote operation and remote communication.

The calculation unit (control unit) 220, which is, for example, a CPU,executes control of the whole guide terminal 200 and various calculationprocesses by executing a control program read from the memory 230. Thedisplay panel 201, which is, for example, a liquid crystal panel,displays, for example, a captured image or map information sent from therobot 100 and/or the participant terminal 300 and images captured by thecamera 205.

The input unit 202 includes a touch panel disposed so as to besuperimposed on the display panel 201 and a push button provided on aperipheral part of the display panel 201. The input unit 202 generatesan operation signal in response to an operation performed by the guideG, and outputs the generated operation signal to the calculation unit220.

The microphone 203 collects voices emitted near the guide terminal 200in accordance with a request from the calculation unit 220, and outputsthe voice signal generated by voice collection to the calculation unit220. The speaker 204 outputs, for example, voices sent from the robot100 and/or the participant terminal 300. The camera 205 captures imagesof, for example, the guide G who is in front of the guide terminal 200(in front of the display panel 201) in accordance with a request fromthe calculation unit 220, and outputs the imaging signal generated byimaging to the calculation unit 220.

The memory 230, which is a non-volatile storage medium, may be, forexample, a solid-state drive. The memory 230 stores, besides a controlprogram for controlling the guide terminal 200, various parametervalues, functions, lookup tables, and the like used for the control andthe calculation.

The communication unit 240, which is, for example, a wireless LAN unit,performs radio communication with the wireless router 600. Thecommunication unit 240 transmits or receives control signals, data orthe like to or from the robot 100 and/or the participant terminals 300via the wireless router 600. For example, the communication unit 240receives captured images or voice signals sent from the robot 100 and/orthe participant terminal 300, map information, or the like and outputsthe received data to the calculation unit 220. Further, thecommunication unit 240 cooperates with the calculation unit 220 andtransmits designation signals, operation signals, captured images, voicesignals or the like to the robot 100 and/or the participant terminal300.

The calculation unit 220 also serves as a functional calculation unitthat executes various processes and calculation. The calculation unit220 includes, for example, an interest analysis unit 221 and a displaycontrol unit 222. The interest analysis unit 221 analyzes interest ofparticipants P based on interest information sent from the participantterminals 300. The display control unit 222 displays, for example, theresult of analyzing interest of the participants P on the display panel201. The display control unit 222 displays information indicating theinterest of the participants P superimposed on the captured image etc.displayed on the display panel 201. It can also be said that the displaycontrol unit 222 and the display panel 201, which cooperate with eachother, configure an output unit that outputs the interest.

FIG. 5 shows a configuration example of the guide screen 210 displayedon the display panel 201 of the guide terminal 200 according to thisembodiment. As shown in FIG. 5 , for example, the guide screen 210includes a robot image display area 211, a map display area 212, a guideimage display area 213, a participant image display area 214, amanipulation panel display area 215, and an interest display area 216.Note that the display form of each area shown in FIG. 5 is merely oneexample, and is not limited thereto.

The robot image display area 211 displays captured images around therobot 100 that have been captured and transmitted by the robot 100.Further, the robot image display area 211 displays, for example, a heatmap in accordance with the interest of the participants P in such a waythat the heat map is superimposed on an image captured by the robot 100.

The map display area 212 displays map information and positionalinformation near the robot 100 transmitted by the robot 100. The mapdisplay area 212 displays, for example, information on, for example, atourist spot near the robot 100, the current location, the moving path,the destination, and the orientation of the robot 100, and the like.

The guide image display area 213 displays images (guide images) thathave been captured by the camera 205 and mainly includes the face of theguide G (manipulator). The participant image display area 214 displayscaptured images (participant images) that have been captured andtransmitted by the plurality of participant terminals 300 and mainlyinclude the faces of the plurality of participants P.

The manipulation panel display area 215 displays a manipulation panelfor manipulating (remotely operating) the robot 100. The manipulationpanel includes a button or the like for adjusting the moving directionor the moving speed of the robot 100, the orientation of the camera andthe like.

The interest display area 216 displays information regarding theinterest of the participants P. The interest display area 216 displays,for example, a level meter indicating the level of interest and a graphindicating a temporal change of interest.

FIG. 6 is a block diagram showing a block configuration example of theparticipant terminal 300 according to this embodiment. As shown in FIG.6 , the participant terminal 300 includes, as functional blocks, acalculation unit 320, the display panel 301, an input unit 302, amicrophone 303, a speaker 304, a camera 305, a memory 330, and acommunication unit 340.

Main elements related to the remote communication with the robot 100,the guide terminal 200, and other participant terminals 300 will bedescribed below. However, the participant terminal 300 may includeelements in its configuration other than the above ones and may includeadditional elements that contribute to the processing for performingremote communication.

The calculation unit (control unit) 320 is, for example, a CPU, andexecutes control of the whole participant terminal 300 and variouscalculation processes by executing a control program loaded from thememory 330. The display panel 301 is, for example, a liquid crystalpanel, and displays, for example, captured images sent from the robot100, the guide terminal 200, or other participant terminals 300, mapinformation, and images captured by the camera 305.

The input unit 302 includes a touch panel disposed so as to besuperimposed on the display panel 301 and a push button provided on aperipheral part of the display panel 301. The input unit 302 generatesan operation signal in response to an operation by the participant P andoutputs the operation signal to the calculation unit 320.

The microphone 303 collects voices emitted near the participant terminal300 in accordance with a request from the calculation unit 320, andoutputs the voice signal generated by voice sound collection to thecalculation unit 320. The speaker 304 outputs, for example, voices sentfrom the robot 100, the guide terminal 200, and other participantterminals 300. The camera 305 captures images of, for example, aparticipant P who is in front of the participant terminal 300 (in frontof the display panel 301) in accordance with a request from thecalculation unit 320, and outputs the imaging signal generated byimaging to the calculation unit 320.

The memory 330, which is a non-volatile storage medium, may be, forexample, a solid-state drive. The memory 330 stores, besides a controlprogram for controlling the participant terminal 300, various parametervalues, functions, lookup tables and the like used for the control andthe calculation.

The communication unit 340, which is, for example, a wireless LAN unit,performs radio communication with the wireless router 600. Thecommunication unit 340 transmits or receives control signals, data orthe like to or from the robot 100, the guide terminal 200, and otherparticipant terminals 300 via the wireless router 600. The communicationunit 340 receives, for example, captured images or voice signals sentfrom the robot 100, the guide terminal 200, and the other participantterminals 300, or the map information, and outputs the received data tothe calculation unit 320. Further, the communication unit 340, whichcooperates with the calculation unit 320, transmits the captured images,the voice signals and the like to the guide terminal 200 and the otherparticipant terminals 300.

The calculation unit 320 also serves as a functional calculation unitthat executes various processes and calculation. For example, thecalculation unit 320 includes an interest detection unit 321 and aninterest notification unit 322. The interest detection unit 321 detectsthe interest of a participant P in accordance with an input operation orthe like performed by the participant P. The interest notification unit322 notifies the guide terminal 200 of the interest informationindicating the detected interest via the communication unit 340.

FIG. 7 shows a configuration example of a participant screen 310displayed on the display panel 301 of the participant terminal 300according to this embodiment. As shown in FIG. 7 , the participantscreen 310 includes, for example, a robot image display area 311, a mapdisplay area 312, an own participant image display area 313, an otherparticipant image display area 314, and a guide image display area 315.Note that the display form in each area shown in FIG. 7 is merely oneexample, and is not limited to them.

The robot image display area 311 displays captured images near the robot100 that have been captured and transmitted by the robot 100. The mapdisplay area 312 displays the map information and the positionalinformation near the robot 100 that have been transmitted from the robot100. The map display area 312 displays, for example, information on, forexample, a tourist spot near the robot 100, the current location, themoving path, the destination, the orientation of the robot 100 and thelike.

The own participant image display area 313 displays an image (an ownparticipant image) that has been captured by the camera 305 and mainlyincludes the face of the own participant P who operates the participantterminal 300. The other participant image display area 314 displaysimages (other participant images) that have been captured andtransmitted by the other participant terminals 300 and mainly includethe faces of the other participants P who operate the other participantterminals 300. The guide image display area 315 displays an image (aguide image) that has been captured and transmitted by the guideterminal 200 and mainly includes the face of the guide G.

FIG. 8 shows an operation example of the remote robot system 10according to this embodiment. As shown in FIG. 8 , the guide terminal200 transmits an operation signal for remotely operating the robot 100to the robot 100 in accordance with the operation by the guide G (S101).In the guide terminal 200, when the guide G operates the manipulationpanel displayed in the manipulation panel display area 215 of the guidescreen 210, the calculation unit 220 generates an operation signalindicating control of moving, imaging, etc. in accordance with theoperation by the guide G, and transmits the operation signal to therobot 100 via the communication unit 240.

Next, the robot 100 performs a moving operation and an imaging operationin accordance with the operation signal (S102) and transmits thepositional information and the captured images to the guide terminal 200and the plurality of participant terminals 300 (S103). In the robot 100,after the communication unit 170 receives the operation signal from theguide terminal 200, the control unit 150 controls driving of themovable-base drive unit 142 and the upper-body drive unit 143 inaccordance with the operation signal that has been received, andcontrols an imaging operation and the like of the camera 131. Thecontrol unit 150 transmits the images captured by the camera 131 and thepositional information (e.g., the latitude and the longitude) detectedby the GPS unit 135 to the guide terminal 200 and the plurality ofparticipant terminals 300 via the communication unit 170. Further, thecontrol unit 150 transmits map information, voices and the like near therobot 100 to the guide terminal 200 and the plurality of participantterminals 300 as necessary.

Note that the positional information and the captured images may betransmitted from the robot 100 to the guide terminal 200 and each of theplurality of participant terminals 300, or the positional informationand the captured images may be transmitted from the robot 100 to theguide terminal 200 and further forwarded from the guide terminal 200 tothe plurality of participant terminals 300.

Next, the guide terminal 200 and the plurality of participant terminals300 output the information received from the robot 100 (S104). In theguide terminal 200, when the communication unit 240 receives thepositional information and the captured images from the robot 100, thecalculation unit 220 (the display control unit 222) causes the robotimage display area 211 of the guide screen 210 to display the receivedimages captured by the robot 100, and causes the map display area 212 todisplay positional information of the robot 100 that has been received.Further, when the guide terminal 200 has received map information fromthe robot 100, the guide terminal 200 updates the map information in themap display area 212. When the guide terminal 200 has received a voicesignal, the guide terminal 200 causes the speaker 204 to output thevoice.

Like in the guide terminal 200, in the participant terminal 300, whenthe communication unit 340 receives the positional information and thecaptured images from the robot 100, the calculation unit 320 causes therobot image display area 311 of the participant screen 310 to displaythe received images captured by the robot 100, and causes the mapdisplay area 312 to display positional information of the robot 100 thathas been received. Further, when the participant terminal 300 hasreceived map information from the robot 100, the participant terminal300 updates the map information in the map display area 312. When theparticipant terminal 300 has received a voice signal, the participantterminal 300 causes the speaker 304 to output the voice.

Next, the plurality of participant terminals 300 detect the interest ofeach of the participants P in accordance with, for example, theoperation by each participant P (S105), and transmits interestinformation indicating the interest that has been detected to the guideterminal 200 (S106). The interest detection unit 321 of the participantterminal 300 detects the interest of a participant P based on, forexample, an operation state, an utterance state, an operation attentionor the like of the participant P. The interest detection unit 321 of theparticipant terminal 300 may detect the interest of the participant Pbased on, for example, one of the operation state, the utterance state,the attention state and the like or based on any given states. After theinterest detection unit 321 detects interest, the interest notificationunit 322 transmits information on the interest (each state) that havebeen detected to the guide terminal 200 via the communication unit 340.

For example, the interest detection unit 321 detects the operation stateof the participant P in response to the display (output) of theparticipant screen 310, and detects the interest of the participant Pbased on the operation state that has been detected. For example, theinterest notification unit 322 may transmit interest informationincluding the operation state that has been detected. The operationstate to be detected includes operation elements such as an operationposition, the number of times of operations, an operation time and thelike in the operation by the participant P on the participant screen 310that displays information on the robot 100. One of the operationelements or any given operation elements may be used. The operation tobe detected may be a touch operation on the touch panel of the displaypanel 301 or may be a mouse pointer movement or a clicking operation bya mouse operation.

When the operation position is detected, a specific position operated onthe participant screen 310 may be detected, or a predetermined areaincluding the operated position may be detected. For example, theparticipant screen 310 may be divided into some areas and a divided areaincluding the operated position may be detected. The interest detectionunit 321 detects the position or the area operated in the robot imagedisplay area 311 or the map display area 312.

Further, when the number of times of operations is detected, the numberof touches or the number of mouse clicks or the like in a predeterminedperiod may be detected. When the operation time is detected, a touchduration time (long press time), a click duration time or the like maybe detected. When, for example, the number of touches is lager or thetouch duration time is longer than a predetermined threshold in apredetermined area, the interest detection unit 321 may detect that thisarea is of great interest (that the participant is excited) and transmitinformation indicating that the area of great interest has beendetected.

Further, the interest detection unit 321 detects the utterance state ofa participant P in response to a display on the participant screen 310,and detects the interest of the participant P based on the utterancestate that has been detected. The interest notification unit 322 maytransmit interest information including the utterance state that hasbeen detected. The utterance state to be detected includes utteranceelements such as a sound pressure of the voice emitted by theparticipant P, an utterance time, and results of recognizing the voice(keyword). One of the utterance elements may be used or any givenutterance elements may be used.

The interest detection unit 321 detects, for example, the sound pressureof the voice of a participant P by the microphone 303. Then, theinterest detection unit 321 may detect that the participant P is excitedby detecting phrases like “Wow!” or “Terrific!”. When the utterance timeis detected, a time during which a predetermined sound pressurecontinues may be detected. For example, the interest detection unit 321may detect that the level of interest of the participant P has beenincreasing when the sound pressure of the participant P is larger than apredetermined threshold or a predetermined sound pressure continues fora predetermined period of time.

Further, the interest detection unit 321 may detect a predeterminedvoice by voice recognition. For example, voice recognition may beperformed on a voice of a participant P detected by the microphone 303.When the interest detection unit 321 has recognized predeterminedphrases like “Wow!” or “Terrific!”, which indicate that the participantP is excited, the interest detection unit 321 may detect that the levelof interest of the participant P has been increasing. In someembodiments, a predetermined keyword related to interest is recognized.This keyword may be the name of a building which is located near therobot 100 or a keyword related to the tourist spot. Further, the wordsto be detected regarding the interest (indicating that the participantis excited) may be negative keywords, not positive keywords. When theparticipant terminal 300 performs remote communication, the utterancestate for the interest may be detected in a state in which the remotecommunication audio is muted.

Further, the interest detection unit 321 may detect the attention stateof a participant P in response to a display on the participant screen310, and detect the interest of the participant P based on the attentionstate that has been detected. The interest notification unit 322 maytransmit interest information including the attention state that hasbeen detected. The attention state that has been detected includesattention elements such as the orientation of the face of theparticipant P, the orientation of the line of sight of the participantP, the attention position, and the attention time. One of the attentionelements may be used or any given attention elements may be used.

The interest detection unit 321 detects, for example, the orientation ofthe face of a participant P by an image of the camera 305, or detectsthe orientation of the line of sight of the participant P by aline-of-sight sensor provided in the participant terminal 300. Theinterest detection unit 321 estimates the attention position (it may bean attention area) on the participant screen 310 from the orientation ofthe face of the participant P or the orientation of the line of sight ofthe participant P. When the attention time is detected, the interestdetection unit 321 may detect time during which a predeterminedorientation of the face, a predetermined orientation of the line ofsight, or a predetermined attention position continues. The interestdetection unit 321 may detect that the level of interest of theparticipant P in this position (orientation) has been increasing when,for example, the same orientation of face, the same orientation of theline of sight, or the same attention position continues for apredetermined period of time. Further, the interest detection unit 321may detect, in addition to or in place of the orientation of the face ofthe participant P, facial expressions of the participant P by imagerecognition. For example, positive and negative facial expressions maybe detected.

Next, the guide terminal 200 displays the interest information receivedfrom the plurality of participant terminals 300 (S107). In the guideterminal 200, when the communication unit 240 receives the interestinformation from the plurality of participant terminals 300, theinterest analysis unit 221 analyzes the interest of the plurality ofparticipants P based on the received interest information and thedisplay control unit 222 displays the results of the analysis on theguide screen 210. The interest analysis unit 221 aggregates orstatistically processes the operation state, the utterance state, theattention state or the like of a participant P, detects the part of thescreen which the participant P is focusing on, or detects a situation inwhich the participant P is excited, and displays the results of thedetection in such a way that the guide G can recognize them. Thestatistical processing includes processing of obtaining a total value,an average value, a median value, a maximum value, a minimum value orthe like, and includes statistics for each position (area) of the screenor statistics for each time. For example, the position of the guidescreen 200 that corresponds to the interest information may behighlighted or a temporal change of the interest information may bedisplayed on the guide screen 210 that displays information on the robot100. As an example of highlighting the position, a heat map may besuperimposed on the robot image display area 211 of the guide screen210. Further, as an example of displaying the temporal change, a levelmeter or a graph may be displayed in the interest display area 216 ofthe guide screen 210. In some embodiments, they are displayed in anaspect in accordance with the degree (level) of interest indicated bythe interest information. The degree of interest is, for example, avalue obtained based on the number of times of operations or theoperation time in the state of operation by a plurality of participants,the sound pressure or an utterance time in the utterance state, and anattention time in the attention state, and each element may be weighted.The results of statistics of all the participants may be collectivelydisplayed or they may be displayed in such a manner that the results ofstatistics for each participant can be recognized. They may be displayedin such a way that the number of participants who show interest can berecognized.

FIG. 9 is an example of displaying results of analyzing the interestinformation by a heat map. As shown in FIG. 9 , the display control unit222 displays, for example, the heat map in accordance with the interestinformation in the robot image display area 211 of the guide screen 210.The heat map may be displayed on the captured image in the robot imagedisplay area 211 or may be displayed on the map information in the mapdisplay area 212. Distribution of the interest is displayed in such away that it is superimposed on parts of the captured image or the mapinformation where the attention of the participant P is focused on(parts where the participant P who sees these parts is excited) so thatthe distribution can be visually recognized. For example, the area ofgreat interest (interest is focused on) is displayed in red, and theheat map is displayed by using different shades and colors according tothe level of attention (interest).

FIG. 10 is an example of displaying the results of analyzing theinterest information by shapes such as circles. As shown in FIG. 10 ,the display control unit 222 causes, for example, the robot imagedisplay area 211 of the guide screen 210 to display circular frames inaccordance with the interest information. The circular frame may bedisplayed on a captured image in the robot image display area 211 or maybe displayed on the map information in the map display area 212. Thecircular frames are displayed so as to surround parts on the capturedimage or the map information where the attention of a participant P isfocused on (parts where the participant P who sees these parts isexcited). For example, an area of great interest (interest is focusedon) is displayed by a red frame, and the frames are displayed by usingdifferent colors, sizes, or the thicknesses according to the level ofattention (the level of interest). Note that the shape of the frames isnot limited to circle and may be any other shape.

FIGS. 11A and 11B are examples of displaying the results of analyzingthe interest information by a level meter. As shown in FIGS. 11A and11B, the display control unit 222 displays, for example, the level meterof the level in accordance with the interest information in the interestdisplay area 216 of the guide screen 210. The level meter displays thelevel (high/low) of the current interest (how excited a participant is).As shown in FIG. 11A, when the current interest level is low, the levelmeter shows a low level. As shown in FIG. 11B, when the current interestlevel is high, the level meter shows a high level. The level of thecurrent interest may be displayed in the level meter in such a way thatdifferent levels are shown by different colors.

FIG. 12 is an example of displaying results of analyzing the interestinformation by a graph. As shown in FIG. 12 , the display control unit222 displays, for example, the graph showing the level of the currentinterest in accordance with the interest information in the interestdisplay area 216 of the guide screen 210. The graph shows a temporalchange regarding how excited a participant is. As shown in FIG. 12 , thelevel of interest is displayed in a line graph in time order to displaya relative change regarding how excited the participant is. The graph isnot limited to a line graph and may be a bar graph or the like. Thegraph may be displayed by using different colors according to the leveland temporal change.

As shown in FIG. 8 , next, the guide terminal 200 transmits theoperation signal for remotely operating the robot 100 to the robot 100in accordance with the operation by the guide G, like in S101 (S108).The guide G knows the interest of the participant P from the interestinformation displayed on the guide screen 210 and operates themanipulation panel of the manipulation panel display area 215 of theguide screen 210 in such a way that the robot 100 moves, for example,toward the place of great interest. The calculation unit 220 generatesan operation signal in accordance with the operation by the guide G, andtransmits the operation signal to the robot 100 via the communicationunit 240. After that, the robot 100 performs the moving operation, theimaging operation and the like in accordance with the operation signal,like in S102.

As described above, in this embodiment, a remote robot system thatimplements a remote sightseeing tour or the like includes a function ofdetecting interest of participants and a function of further presentingthe detected results to the connection terminal of the guide.Accordingly, even when the guide and the participants are remotely awayfrom each other, it is possible to send feedback information indicatingwhich part the participants are focusing on and which part theparticipants are interested in to the guide. This enables the guide toexplain the part the participants are interested in and enables toobtain detailed information by moving the robot. Accordingly, it ispossible to provide a tour that meet the needs of participants whilerecognizing reactions of the participants (users), and thus respondappropriately according to participants.

Second Embodiment

Next, a second embodiment according to the present disclosure will bedescribed. In this embodiment, an example of automatically controllingthe operation of the robot in accordance with participants' interest inthe remote robot system according to the first embodiment will bedescribed.

FIG. 13 is a block diagram showing a functional configuration example ofthe guide terminal 200 according to this embodiment. As shown in FIG. 13, the guide terminal 200 includes a control method determination unit223 in the calculation unit 220. The other configurations are similar tothose in the first embodiment. The control method determination unit(remote control determination unit) 223 determines a method ofcontrolling the robot 100 (remote operation method) based on interestinformation sent from the participant terminal 300, that is, inaccordance with interest of a participant P analyzed by the interestanalysis unit 221.

FIG. 14 shows an operation example of the remote robot system 10according to this embodiment. In FIGS. 14 , S101 to S107 are similar tothose in the first embodiment. After S107, the guide terminal 200determines the method of controlling the robot 100 in accordance withinterest information of a participant P (S110), and transmits anoperation signal of the control method that has been determined to therobot 100 (S108). When the interest analysis unit 221 analyzes interestinformation of the participant P and the display control unit 222displays the results of the analysis, the control method determinationunit 223 determines the method of controlling the robot 100 inaccordance with the interest information of the participant P that hasbeen analyzed. Note that the display of the interest information may beomitted and the control method may be determined in S110.

The control method determination unit 223 determines, for example, amoving method of the robot 100 including the moving direction, themoving path, the moving speed and the like of the robot 100. The controlmethod determination unit 223 may automatically select a path of greatinterest from a plurality of tour paths in accordance with the interestinformation that has been analyzed. For example, when there are manyparticipants P who are looking at a specific building, a path thatpasses near this building is selected. Further, the control methoddetermination unit 223 may control the moving speed of the robot 100 inaccordance with the interest information that has been analyzed. Forexample, the speed of the robot 100 may be adjusted depending on themagnitude of the degree of interest in such a way that the viewing timefor areas of interest becomes longer and the viewing time for areas ofless interest becomes shorter. The speed at which the robot 100 movesmay be reduced or the robot 100 may stop moving in areas of interest.

Further, the control method determination unit 223 may determine aninformation acquisition method of the robot 100 including, for example,an imaging direction in which the robot 100 captures images or a voicecollecting direction. The control method determination unit 223 maycontrol the imaging direction of the camera 131 of the robot 100 in sucha way that the camera 131 automatically faces the direction of interestin accordance with the interest information that has been analyzed, ormay zoom in the imaging range of the camera 131. Likewise, the controlmethod determination unit 223 may control the collection direction ofthe microphone 132 in such a way that the microphone 132 faces thedirection of interest.

The robot 100 may be automatically controlled by transmitting anoperation signal in accordance with the control method that has beendetermined or may display the control method that has been determined onthe guide screen 210 to recommend the moving path and the like of therobot 100 to the guide G. Further, the determination of the controlmethod and the execution of the control may not be performed by theguide terminal 200 and may be performed by another apparatus.

As described above, in this embodiment, the method of controlling therobot is determined in accordance with participant's interest in aremote robot system. Accordingly, the robot can be automaticallycontrolled in accordance with the participant's interest, whereby it ispossible to respond appropriately according to participants.

Note that the present disclosure is not limited to the above embodimentsand may be changed as appropriate without departing from the spirit ofthe present disclosure. For example, while the example in which theremote robot system is used for a remote sightseeing tour has beendescribed in the above embodiments, the remote robot system may be usedfor applications other than the remote sightseeing tour. This remoterobot system may be used, for example, to remotely search for missingpersons such as lost children or to remotely check the states of familymembers. The above embodiments can be efficiently used when multipleremote users remotely operate a robot and don't know where to focus onwhen remotely controlling the robot.

Each of the configurations in the above embodiments may be constructedby software, hardware, or both of them. Further, each of theconfigurations may be formed by one hardware device or one softwareprogram, or a plurality of hardware devices or a plurality of softwareprograms. The function (the process) of each apparatus may beimplemented by a computer including a CPU, a memory and the like. Forexample, a program for performing a method (control method) according tothe embodiments may be stored in a storage device, and each function maybe implemented by having the CPU execute the program stored in thestorage device.

The program includes instructions (or software codes) that, when loadedinto a computer, cause the computer to perform one or more of thefunctions described in the embodiments. The program may be stored in anon-transitory computer readable medium or a tangible storage medium. Byway of example, and not limitation, non-transitory computer readablemedia or tangible storage media can include a random-access memory(RAM), a read-only memory (ROM), a flash memory, a solid-state drive(SSD) or other memory technologies, CD-ROM, digital versatile disc(DVD), Blu-ray (Registered Trademark) disc or other optical discstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices. The program may be transmitted on atransitory computer readable medium or a communication medium. By way ofexample, and not limitation, transitory computer readable media orcommunication media can include electrical, optical, acoustical, orother form of propagated signals.

From the disclosure thus described, it will be obvious that theembodiments of the disclosure may be varied in many ways. Suchvariations are not to be regarded as a departure from the spirit andscope of the disclosure, and all such modifications as would be obviousto one skilled in the art are intended for inclusion within the scope ofthe following claims.

What is claimed is:
 1. A remote robot system comprising: a robotconfigured to perform a predetermined operation including collection oflocal information near the robot; a first remote terminal capable ofremotely operating the operation of the robot; and a second remoteterminal capable of remotely communicating with the first remoteterminal, wherein the second remote terminal comprises: an informationoutput unit configured to output the local information acquired from therobot to a user; and an interest detection unit configured to detectuser's interest in the output local information.
 2. The remote robotsystem according to claim 1, wherein the interest detection unit detectsan operation state by the user in response to the output localinformation and detects the user's interest based on the operation statethat has been detected.
 3. The remote robot system according to claim 2,wherein the operation state includes an operation position, the numberof times of operations, or an operation time in the user's operationperformed on a display screen that displays the local information. 4.The remote robot system according to claim 1, wherein the interestdetection unit detects an utterance state of the user in response to theoutput local information and detects the user's interest based on theutterance state that has been detected.
 5. The remote robot systemaccording to claim 4, wherein the utterance state includes a soundpressure of a voice emitted by the user, an utterance time of the voice,or results of recognizing the voice.
 6. The remote robot systemaccording to claim 1, wherein the interest detection unit detects anattention state of the user in response to the output local informationand detects the user's interest based on the attention state that hasbeen detected.
 7. The remote robot system according to claim 6, whereinthe attention state includes the orientation of the face of the user,the orientation of the line of sight of the user, an attention positionof the user, or an attention time of the user.
 8. The remote robotsystem according to claim 1, wherein the second remote terminalcomprises a notification unit configured to notify the first remoteterminal of interest information indicating the detected interest. 9.The remote robot system according to claim 8, wherein the first remoteterminal comprises an interest output unit configured to output theinterest information that has been sent.
 10. The remote robot systemaccording to claim 9, wherein the interest output unit highlights aposition that corresponds to the interest information on a displayscreen that displays the local information.
 11. The remote robot systemaccording to claim 9, wherein the interest output unit displays atemporal change of the interest information on a display screen thatdisplays the local information.
 12. The remote robot system according toclaim 8, wherein the first remote terminal comprises a remote controldetermination unit configured to determine a method of remotelyoperating the robot based on the interest information that has beensent.
 13. The remote robot system according to claim 12, wherein theremote control determination unit determines a method of moving a robotincluding a moving direction, a moving path, or a moving speed of therobot based on the interest information.
 14. The remote robot systemaccording to claim 12, wherein the remote control determination unitdetermines an information acquisition method of the robot including animaging direction of an image of the robot or a voice collectiondirection based on the interest information.
 15. A method of controllinga remote robot system comprising: a robot configured to perform apredetermined operation including collection of local information nearthe robot; a first remote terminal capable of remotely operating theoperation of the robot; and a second remote terminal capable of remotelycommunicating with the first remote terminal, wherein the second remoteterminal outputs the local information acquired from the robot to auser, and the second remote terminal detects user's interest in theoutput local information.